High Expressive Spatio-Temporal Relations

The management of images remains a complex task that is currently a cause for several research works in various domains such as GIS, medicine, etc. In line with this, we are interested in this work with the problem of image retrieval that is mainly related to the complexity of image representation. In this paper, we address the problem of relation representation and we propose a novel method for identifying and indexing relations between salient objects. Our method is based on defining several intersection sets for each feature like shape, time, etc. Relations are then computed using a binary intersection matrix between sets. Several types of relations can be calculated using our method in particular spatial and temporal relations. We show how our method is able to provide high expressive power and homogeneous representation to relations in comparison to the traditional methods. Introduction During the last decade, a lot of work has been done in information technology in order to integrate image retrieval in the standard data processing environments. Image retrieval is involved in several domains [(Yoshitaka and Ichikawa 1999), (Rui, Huang, and Chang 1999), Grosky 1997), (Smeulders, Gevers, and Kersten, 1998)] such as Geographic Information Systems, Medicine, Surveillance, etc. where queries criteria are based on different types of features. Principally, spatio-temporal relations are very important in image retrieval and need to be considered in several domains. In medicine, for instance, the spatial data in surgical or radiation therapy of brain tumors is decisive because the location and the temporal evolution of a tumor has profound implications on a therapeutic decision (Chbeir and Favetta 2000). Hence, it is crucial to provide a precise and powerful system to express spatio-temporal relations required in image retrieval. In this paper, we address this issue and propose a novel method that can easily compute several types of relations with better expressions. The rest of this paper is organized as follows. In section 2, we present the related work. Section 3 is devoted to define our method for calculating relations. Finally, conclusions and future orientations are given in section 4. Related Work Relations between either salient objects, shapes, points of interests, etc. have been widely used in image indexing such as R-tree (Guttman 1984), R+-tree (Sellis, Roussopoulos, and Faloutsos 1987), R*-tree (Beckmann 1990), hB-tree (Lomet and Salzberg 1990), ss-tree (White and Jain 1996), TV-tree (Lin, Jagadish, and Faloutsos 1994), 2D-String (Chang, Shi, and Yan 1987), 2D-G String (Chang and Jungert 1991), 2D-C+ String (Huang and Jean 1994], θR-String (Gudivada 1995], σ-tree (Chang and Jungert 1997), etc. Temporal and spatial relations are the most used relations in image indexing. To calculate temporal relations, Allen relations (Allen 1983) are often used. Allen proposes 13 temporal relations (Before, After, Meet, Touched By, Started by, Overlapped By, Start With, Started With, Finish wish, Finishes, Contain, During, Equal) in which 6 are symmetrical. On the other hand, three major types of spatial relations are generally proposed in image representation (Egenhofer, Frank, and Jackson 1989): Metric relations: measure the distance between salient objects (Peuquet 1986). For instance, the metric relation “far” between two objects A and B indicates that each pair of points Ai and Bj has a distance grater than a certain value δ. Directional relations: describe the order between two salient objects according to a direction, or the localisation of salient object inside images (El-kwae and Kabuka 1999). In the literature, fourteen directional relations are considered: north, south, east, west, north-east, north-west, south-east, south-west, left, right, up, down, front and behind. Topological relations: describe the intersection and the incidence between objects [(Egenhofer and Herring 1991), (Egenhofer 1997)]. Egenhofer (Egenhofer and FLAIRS 2002 471 Copyright © 2002, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. From: FLAIRS-02 Proceedings. Copyright © 2002, AAAI (www.aaai.org). All rights reserved. Herring 1991) has identified six basic relations: Disjoint, Touch, Overlap, Cover, Contain, and Equal. In spite of all the proposed work to represent complex visual situations, several shortcomings exist in the methods of spatial relation computations. For instance, Figure 1 shows two different spatial situations of three salient objects that are described by the same spatial relations in both cases, in particular topological relations: a1 Touch a2, a1 Touch a3, a2 Touch a3; and directional relations: a1 Above a3, a2 Above a3, a1 Left a2. Figure 1: Two different spatial situations. In this paper, we present our method able to provide a high expressive power to represent several types of relations in particular spatial and temporal ones. Proposition Our proposal represents an extension of the 9-Intersection model (Egenhofer and Herring 1991). It provides a general method for computing not only topological relations but also other types of relations in particular temporal and spatial relations with higher precision. The idea is to identify relations in function of two values of a feature such as shape, position, time, etc. The shape feature gives spatial relations, the time feature gives temporal relations, and so on. To identify a relation between two values of the same feature, we propose the use of an intersection matrix between several sets defined in function of each feature: Definition Let us first consider a feature F. We define its intersection sets as follows: The interior F: contains all elements that cover the interior or the core of F. In particular, it contains the barycentre of F. The definition of this set has a great impact on the other sets. F may be empty (∅). The boundary ∂F: contains all elements that allow determining the frontier of F. ∂F is never empty (¬∅). The exterior F: is the complement of F∪∂F. It contains at least two elements ⊥ (the minimum value) and ∞ (the maximum value). F can be divided into several disjoint subsets. This decomposition depends on the number of the feature dimensions. For instance, if we consider a feature of one dimension i (such as the acquisition time of a frame in a video), two intersection subsets can be defined (Figure 2): Fi (or inferior): contains elements of F that do not belong to any other intersection set and inferior to ∂F elements on the basis of i dimension. Fi (or superior): contains elements of F that do not belong to any other intersection set and superior to ∂F elements on the basis of i dimension.